PhyCV: The First Physics-inspired Computer Vision Library

PhyCV is the first computer vision library which utilizes algorithms directly derived from the equations of physics governing physical phenomena. The algorithms appearing in the current release emulate, in a metaphoric sense, the propagation of light through a physical medium with natural and engineered diffractive properties followed by coherent detection. Unlike traditional algorithms that are a sequence of hand-crafted empirical rules or deep learning algorithms that are usually data-driven and computationally heavy, physics-inspired algorithms leverage physical laws of nature as blueprints for inventing algorithms. PhyCV features low-dimensionality and high- efficiency, making it ideal for edge computing applications. We demonstrate real-time video processing on NVIDIA Jetson Nano using PhyCV. In addition, these algorithms have the potential to be implemented in real physical devices for fast and efficient computation in the form of analog computing. The open-sourced code is available at https://github.com/JalaliLabUCLA/phyc

VEViD: Vision Enhancement via Virtual Diffraction and Coherent Detection

We discuss a new paradigm wherein physical phenomena coded as algorithms perform computational imaging tasks. Vision Enhancement via Virtual diffraction and coherent Detection (VEViD) is a high-performance low-light-level and color enhancement tool that emerges from this paradigm. VEViD reimagines a digital image as a spatially varying metaphoric “lightfield” and subjects this field to physical processes akin to diffraction and coherent detection. The term “Virtual” captures the deviation from the physical world. The lightfield is pixelated and the propagation imparts a phase with dependence on spatial frequency. This phase, as opposed to intensity, represents the output image. The algorithm is extremely fast, interpretable, and reduces to a compact and intuitively-appealing mathematical expression. We demonstrate image enhancement of 4K resolution video at over 200 frames per second and show the utility of this physical algorithm in improving the accuracy of object detection in low-light conditions by neural networks